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Executive Summary 


The explosion of knowledge and the growing complexity of modern life are changing expectations 
for what young people need to learn and be able to do. Success in the 21st century does not depend 
solely on what people know, but also on what they can do with what they know. Thus, young people 
need to be able to think critically, collaborate effectively, communicate clearly, solve complex 
problems, and continue to learn independently throughout their lives. To equip the next generation 
of Californians with these skills, the state adopted new learning standards and assessments that 
require all students to engage in higher order thinking and problem-solving. Around the same 
time, California implemented a new funding and accountability system, the Local Control Funding 
Formula, which allocated funds based on pupil needs and removed most categorical restrictions 

on spending. 


These shifts to more challenging standards revealed even wider achievement gaps in many districts 
on the new statewide assessments. Despite wide achievement gaps across the state between 
students from different racial and socioeconomic backgrounds, some California school districts 
have excelled at supporting the learning of all students in this new era of deeper learning. We refer 
to these California school districts as “positive outliers” because their students are beating the odds. 
In these districts, students of color, as well as White students, consistently achieve at higher than 
expected levels, performing better than students of similar racial/ethnic backgrounds from families 
of similar income and education levels in most other California districts. Positive outlier districts 
appear to have leveraged the state’s updated educational standards, funding, and accountability 
systems to support students in meeting the more rigorous academic standards. 


In this report, we summarize the results of a quantitative analysis that identifies districts in 

which students of color, as well as their White peers, have demonstrated extraordinary levels of 
academic achievement, measured by California’s new assessments in English language arts and 
mathematics, taking into account race and family income and education levels. These results show, 
for the first time, which California districts and communities appear to have best supported the 
academic achievement of students in the first 3 years of the new assessments, controlling for the 
socioeconomic status of families in each district. We also examine some of the factors associated 
with their success. 


We find that, aside from socioeconomic status, a major predictor of student achievement is the 
preparedness of teachers. The proportion of teachers holding less than a full credential (i.e., an 
intern credential, temporary or short-term permit, or waiver for their teaching position) shows a 
strong negative association with student achievement for all student groups. In addition, teachers’ 
average experience levels are positively associated with achievement for African American and 
Hispanic students. We recognize that these teacher qualifications are also associated with other 
variables that influence staff recruitment and retention and may signal broader differentials in 
teaching and learning conditions. California districts that have been able to find and keep fully 
prepared teachers have supported stronger student achievement for African American and Hispanic 
students as well as for White students. 
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Introduction 


The rapid explosion of knowledge and the growing complexity of modern life are changing 
expectations for what young people need to learn and be able to do. Success in the 21st century 
does not depend solely on what people know, but also on what they can do with what they know to 
solve complex problems and continue to learn independently throughout their lives. To respond to 
these new realities, California adopted the Common Core State Standards and the Next Generation 
of Science Standards, which require all students to engage in the kind of higher order thinking and 
problem-solving once reserved for a small minority. Implementing the standards requires principals 
and teachers to shift instructional practices from those geared to recalling information to those 
aimed at applying knowledge to complex problems and using evidence, inquiry, and multiple modes 
of communication. 


In response to the updated learning standards, California developed new assessments in English 
language arts and mathematics through the Smarter Balanced Assessment Consortium, a multistate 
collaborative. Now called the California Assessment of Student Performance and Progress 
(CAASPP), these assessments, like the standards, focus on higher order thinking and performance 
skills and include open-ended items as well as inquiry-based performance tasks in both subject 
areas; these require young people to research a critical issue or take up a complex problem and 
write extended explanations of their analysis and conclusions. 


These standards and assessments are significantly different from the earlier standards and 
multiple-choice tests that were used to guide instruction. They now allow the state to assess 
the kind of deeper learning that is a prerequisite to success in today’s knowledge-based society 
and economy. 


Around the same time as the implementation of Common Core and CAASPP, California 
implemented a new funding and accountability system, the Local Control Funding Formula (LCFF). 
LCFF gave districts more autonomy to decide how to allocate their state funding, while providing 
increased funding on behalf of high-need students (i.e., English learners, students from low-income 
families, and foster youth). 


Although these changes were in part motivated by the desire to improve the achievement of 
historically underserved students, achievement gaps in many districts have widened on statewide 
assessments. The more advanced skills now measured in the new assessments have often been 
reserved for the most advantaged learners—a tendency that was exacerbated in the earlier era of 
accountability, which focused on lower level skills and test preparation aimed at multiple-choice 
tests rather than the inquiry, writing, and problem-solving now encouraged. Studies found that 
instruction in districts serving high-need students, where sanctions were most often threatened, 
emphasized lower level skills at the expense of a broader curriculum.' 
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Despite wide achievement gaps across the state 

between students from different racial and Despite wide achievement 
socioeconomic backgrounds,” some California gaps across the state between 
school districts have excelled at supporting the 
learning of all students in this new era of deeper 
learning. We refer to these California school and socioeconomic backgrounds, 
districts as “positive outliers” because their students in some districts are 
students are beating the odds relative to the 
socioeconomic conditions in their communities. 
In these districts, students of color, as well as socioeconomic conditions in their 
White students, consistently achieve at higher 
than expected levels, performing better than 
students of similar racial/ethnic backgrounds California school districts as 
from families of similar income and education “positive outliers.” 

levels in most other California districts. Positive 

outlier districts appear to have leveraged 

the state’s updated educational standards, 

funding, and accountability systems to support students of color in meeting the more rigorous 
academic standards. 


students from different racial 


beating the odds relative to the 


communities. We refer to these 


In this paper, we summarize the results of a quantitative analysis that identifies districts in which 
students of color, as well as their White peers, have demonstrated extraordinary levels of academic 
achievement, measured by California’s new assessments in English language arts and mathematics, 
taking into account race, family income, and education levels. Acknowledging that students’ 
achievements may be a result of school, community, and/or family factors, we have conducted 

an analysis of school district inputs and resources associated with this higher than predicted 
achievement. These results show, for the first time, which California districts and communities 
appear to have best supported the academic achievement of students in the first 3 years of the new 
assessments, controlling for the socioeconomic status (SES) of families in each district. We also 
examine some of the factors associated with their success. 


Our analysis proceeds in two parts. In the first part, we identify the California districts that 

have been particularly successful at supporting the achievement of African American, Hispanic, 
and White students.* The second part of our analysis examines the factors that predict student 
achievement for African American, Hispanic, and White students at the district level. We describe 
our methods for each of these analyses below. 
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Part I: Identification of Positive Outlier Districts 


Methodology 


Data and variables 


To estimate student achievement, we use district-level mathematics and English language arts 
(ELA) results from the CAASPP. We use the CAASPP results for all students and for three racial/ 
ethnic subgroups (African American, Hispanic, and White) for 2015, 2016, and 2017. In our analyses, 
we exclude districts in which fewer than 200 African American or Hispanic students and 200 

White students were tested, so that the estimates of socioeconomic characteristics for each group 
are sufficiently reliable. Consequently, our initial analyses include 435 California districts. These 
districts account for approximately 42% of the districts in California and for 86% of California’s 
public school students, including 88% of Hispanic, 91% of African American, and 81% of White 
public school students in the state. 


Mean test scores by year, subject, and grade level for all students and for these racial subgroups 
(African American, Hispanic, and White) in 2015, 2016, and 2017 are standardized using the year- 
grade-subject specific state mean and standard deviation. 


We supplement California Department of Education data with additional information about each 
school district. To estimate the socioeconomic conditions of students from each racial/ethnic group, 
within each school district, we use six measures of socioeconomic characteristics for families with 
children attending public school. These measures are from the American Community Survey’s (ACS) 
Education Demographic and Geographic Estimates (EDGE). These socioeconomic variables are 
available by racial/ethnic subgroup as well. The six measures include each district’s 


e median income; 

e unemployment rate; 

e proportion of households who participate in the Supplemental Nutrition Assistance 
Program (SNAP), formerly the Food Stamp Program; 

e proportion of parents with a bachelor’s degree or higher; 

e proportion of households with children age 5 to 17 in poverty; and 

e proportion of households headed by single mothers. 


We also include a measure of the proportion of socioeconomically disadvantaged students by 

race tested in each school district, as reported by the California Department of Education. To be 
considered socioeconomically disadvantaged, students must meet at least one of the two criteria: 
neither of their parents received a high school diploma, or they are eligible to receive free or 
reduced-price lunch. Data also include the total number of students enrolled as well as the number 
of students tested in each district, grade, subgroup, and subject. The names and urbanicity of 
districts come from the Common Core of Data. 


We use a two-level precision-weighted hierarchical linear model to identify the districts whose 
students in each racial/ethnic group (African American, Hispanic, and White) are performing better 
on CAASPP than one would predict based on the SES of the families each district serves. We use a 
multilevel model because we observe average test scores in multiple grades, years, and subjects in 
each district; thus, the model accounts for the nesting of grade-year-subject cells within districts. 
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The precision weighting gives more weight in the estimation to cells whose mean test scores are 
more precisely estimated (that is, it gives more weight to observations based on larger numbers 
of students). 


The models tell us the difference between the actual average performance of a district’s students in a 
given racial subgroup and what one would predict the performance to be of the district’s students in 
the given subgroup based on the district’s socioeconomic characteristics—a calculation known as the 
residual. In all cases, a larger positive residual means a district performed better than expected for 
students of a given racial/ethnic group, controlling for the SES of families of that group in the district. 
A larger negative residual means a district performed worse than expected. Figure 1 illustrates the 
basic model for calculating residuals. We rank districts based on their value of the residual. 


For a more detailed explanation of the methodology, see Appendix A. 


Figure 1 
Understanding Residuals 


Predicted Student 


Actual Student Student 
Subgrou Udell» = Subgrou 

; ereue Achievement Based g P 
Achievement Residual 


on District SES 


Results 


Significant variation in California student achievement 


In many districts across the state, African American, Hispanic, and White students are achieving 
ELA and math scores that are higher than expected given the socioeconomic conditions for each 

of those groups in their communities. Figure 2 shows the variation in student achievement across 
California’s 435 districts with at least 200 African American or Hispanic students and 200 White 
students. Observations in the top right quadrant of Figure 2 are considered positive outlier districts 
because African American and Hispanic students, as well as White students, achieve at higher than 
predicted levels based on their socioeconomic status. California has 156 districts in which students 
achieve at much higher than expected levels. In contrast, districts in the lower left quadrant are 
underperforming because students of all racial/ethnic groups achieve at lower levels than predicted 
by their socioeconomic status. 


In Figure 2, positive observations along the x-axis show how many standard deviations higher 
White students in the district achieve than would be predicted based on the SES of White families 

in the district. Negative observations along the x-axis show how many standard deviations lower 
White students in the district achieve than would be predicted based on White families’ SES in the 
district. Similarly, positive observations along the y-axis show how many standard deviations higher 
African American and Hispanic students in the district achieve than would be predicted based on 
the SES of African American and Hispanic families in the district. 
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In this plot, we average each subgroup’s performance across subjects (i.e., math and ELA), tested 
grades (i.e., grades 3 through 8 and 11), and years (i.e., 2015, 2016, and 2017). Because some California 
districts have limited grades of students (e.g., elementary districts only have tested students in grades 
3 through 8), we average the performance across the available grades for each district. We also combine 
the African American and Hispanic subgroup results, weighted by the number of African American 
versus Hispanic students in the district, to reflect the achievement of students of color in the district. 


In addition, we find that positive outlier districts tend to appear as top districts across years and 
subjects fairly consistently. The correlation between district residuals across subjects and years 
for African American students is approximately .70. For Hispanic students, the correlation is 
approximately .90. Moreover, our estimates are highly reliable; that is, the standard errors of our 
estimates are limited. Thus, we trust that our results capture real differences across districts. (See 
Appendix A for more detail.) 


Figure 2 


Student Achievement in California Districts 
Average African American/Hispanic and White achievement by district averaged 
across subjects, grades, and years (2015, 2016, and 2017) 
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Notes: Figure includes districts with at least 200 African American or Hispanic students and 200 White students. The size 
of the marker is weighted by the number of African American and Hispanic students tested in the district. Achievement is 
measured by residuals in standard deviations. The origin (0,0) represents districts in which African American and Hispanic 
and White students perform as predicted based on the SES of each group’s families in the district. 

LPI analysis of data from California Department of Education. (n.d.). California Assessment of Student Performance and 
Progress (CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. 
(n.d.). Education demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 


LEARNING POLICY INSTITUTE | CALIFORNIA’S POSITIVE OUTLIERS: DISTRICTS BEATING THE ODDS 5 


Some California districts have especially strong African American student achievement on the first 3 
years of CAASPP, while others have especially strong Hispanic student achievement. Many districts in 
California serve a much larger population of one of these groups than the other (typically Hispanic). 
Figure 3 includes the results of zooming in to the top right quadrant of a figure such as Figure 2, which 
includes just African American students. We identify the 48 districts in which both African American 
and White students achieve on average at higher than predicted levels. The x-axis shows how many 
standard deviations higher White students in the district achieve than would be predicted based on the 
SES of White families in the district. Similarly, the y-axis shows how many standard deviations higher 
African American students achieve than would be predicted based on the SES of African American 
families in the district. As in Figure 2, we average each subgroup’s performance across subjects (i.e., 
math and ELA), grades (i.e., grades 3 through 8 and 11), and years (i.e., 2015, 2016, and 2017). 


On our measure, Chula Vista Elementary District is the top district in which both White and African 
American students perform higher than predicted, with several other elementary districts right 
behind. San Diego Unified and Long Beach Unified are the largest districts in which both groups 
outperform expectations by more than .10 standard deviations. See Appendix B for a list of all the 
districts and their residuals in Figure 3. 


Figure 3 
Districts With Higher Than Predicted African American and White 
Student Achievement 


Average African American and White residuals by district averaged across subjects, 
grades, and years (2015, 2016, and 2017) 
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Note: Size of marker is weighted by number of African American students tested in the district. 


Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 
(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. (n.d.). 
Education demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 
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Approximately 167 California districts have had Hispanic and White students consistently achieve on 
average at higher than predicted levels on the first 3 years of CAASPP. Figure 4 shows the results of 
zooming in to the top right quadrant of a figure such as Figure 2, which includes just Hispanic and White 
students. As in Figure 3, the x-axis shows how many standard deviations higher White students in the 
district achieve than would be predicted based on the SES of White families in the district. Similarly, the 
y-axis shows how many standard deviations higher Hispanic students achieve than would be predicted 
based on the SES of Hispanic families in the district. As in Figures 2 and 3, we averaged each subgroup’s 
performance across subjects (i.e., math and ELA), grades (i.e., grades 3 through 8 and 11), and years (i.e., 
2015, 2016, and 2017). Due to space constraints, we only label districts in which either Hispanic students 
or White students achieved at least 0.20 standard deviations higher than predicted. The small districts of 
Newhall, Magnolia Elementary, and Winton are highest performing for both groups. Hawthorne is high 
performing for Hispanics. Chula Vista makes the list once again, as do the larger districts of Long Beach 
and San Diego. See Appendix B for a list of all the districts and their residuals in Figure 4. 


Figure 4 


Districts With Higher Than Predicted Hispanic and White 
Student Achievement 


Average Hispanic and White residuals by district averaged across subjects, 
grades, and years (2015, 2016, and 2017) 
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Notes: Size of marker is weighted by number of Hispanic students tested in the district. Due to space constraints, we only 
label districts with over 25,000 students or districts in which either Hispanic students or White students achieved at least 
0.20 standard deviations higher than predicted. 


Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 
(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. (n.d.). 
Education demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 
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Students of color in over 50 larger California districts consistently beat the odds 


In which California districts have students of color consistently achieved much higher than 
predicted? We identify 54 districts of significant size in which this is the case (see Table 1). To 
identify these districts, we apply a series of filters. First, we limit the sample to districts with 

at least 2,000 students to ensure that our estimates are sufficiently reliable.* Next, we require 
that the district appear as one of the top 50 California districts having higher than predicted 
achievement given the SES of the subgroup in the district in at least 50% of the observations 
for each eligible year, subject, and race combination for African American or Hispanic students. 
For example, if a district has over 200 African American students and 200 Hispanic students, we 
require that the district appear as a top 50 California district in at least six of the 12 possible 
observations. In this case, the 12 possible observations include, for African American students, 
(1) 2015 ELA, (2) 2015 math, (3) 2016 ELA, (4) 2016 math, (5) 2017 ELA, and (6) 2017 math, and, 
for Hispanic students, (7) 2015 ELA, (8) 2015 math, (9) 2016 ELA, (10) 2016 math, (11) 2017 ELA, 
and (12) 2017 math. 


Table 1 
California Positive Outlier Districts 


African % African High-Achieving Group 
District Student American American Hispanic % Hispanic | AA = African American; 
Name Urbanicity | Enrollment | Enrollment | Enrollment | Enrollment | Enrollment | H = Hispanic; W = White 
ia a 20,998 1,846 9% 9,428 45% AA, H, W 
nee ae 19,390 756 4% 15,220 78% AA, W 
pEwatee |) ne 4,855 112 2% 3,390 70% H, W 
Elementary Midsize 
caifeue lecaeo 3,959 25 1% enseks 94% H, W 
Carlsbad City: 
Unified | Midsize 11,049 189 2% 2,904 26% H, W 
Carmel Suburb: 9 9 
Unified Midsize 2,492 13 1% 457 18% H, W 
conte || Sapuls 4,491 140 3% 2,422 54% H, W 
Elementary Large 
sae at cee 29,806 1,051 4% 20,594 69% AA, H, W 
Sa ee 41,169 4,313 3% 14,372 35% AA, H, W 
pales ee 28,999 493 2% 20,949 72% AA, H, W 
iat eee 22,698 650 3% 20,002 88% AA, H, W 
ae eee 3,896 204 5% 2,958 76% H, W 
East Whittier Suburb: 9,064 88 1% 7,470 32% H, W 
City Elementary Large 
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District 
Name 


El Monte 
City 


Urbanicity 


Suburb: 
Large 


Student 
Enrollment 


African 
American 
Enrollment 


% African 
American 
Enrollment 


Hispanic 
Enrollment 


% Hispanic 
Enrollment 


High-Achieving Group 
AA = African American; 
H = Hispanic; W = White 


Encinitas Union 
Elementary 


Etiwanda 
Elementary 


Suburb: 
Large 


Suburb: 
Large 


Eureka 
Union 


Fruitvale 
Elementary 


Suburb: 
Large 


Greenfield 
Union 


Gridley 
Unified 


Town: 
Distant 


Hawthorne 


Irvine 
Unified 


Suburb: 
Large 


City: 
Midsize 


Kerman 
Unified 


Kings Canyon 
Joint Unified 


Town: 
Fringe 


Town: 
Distant 


La Canada 
Unified 


La Mesa- 
Spring Valley 


Suburb: 
Large 


Suburb: 
Large 


Lawndale 
Elementary 


Lemon 
Grove 


Suburb: 
Large 


Lincoln 
Unified 


Little Lake 
City Elementary 


Long Beach 
Unified 


Magnolia 
Elementary 


City: 
Large 


Monrovia 
Unified 


Newhall 


Suburb: 
Large 


Suburb: 
Large 


Nuview 
Union 
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Suburb: 
Large 


District 
Name 


Paramount 
Unified 


Urbanicity 


Suburb: 
Large 


Student 
Enrollment 


African 
American 
Enrollment 


% African 
American 
Enrollment 


Hispanic 
Enrollment 


% Hispanic 
Enrollment 


High-Achieving Group 
AA = African American; 
H = Hispanic; W = White 


Perris 
Elementary 


Redlands 
Unified 


Suburb: 
Large 


Riverside 
Unified 


Rocklin 
Unified 


Suburb: 
Large 


Roseland 


Rosemead 
Elementary 


Suburb: 
Large 


Suburb: 
Large 


San Bernardino 
City Unified 


San Diego 
Unified 


City: 
Midsize 


City: 
Large 


1295779 


12,085 


60,884 


San Marcos 
Unified 


Sanger 
Unified 


Suburb: 
Large 


Town: 
Fringe 


20,452 


455 


9,365 


Santa Clara 
Unified 


Santa Monica- 
Malibu Unified 


City: 
Midsize 


Solana Beach 
Elementary 


Sulphur Springs 
Union 


Suburb: 
Large 


Upland 
Unified 


Vacaville 
Unified 


Suburb: 
Large 


Suburb: 
Small 


Val Verde 
Unified 


Weaver 
Union 


Suburb: 
Large 


City: 
Small 


Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 


(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Stati 


demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 
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stics. (n.d.). Education 
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The 54 positive outlier districts are scattered throughout the state. Figure 5 includes a map of 

these districts. The magenta dots on the map represent positive outlier school districts, and the 
white dots represent major cities for reference. Because we limit our analysis to districts with over 
2,000 students, a restriction that removed many districts in less populated communities, a larger 
proportion of positive outlier districts are in urban and suburban areas than in rural and town areas. 
In addition, few positive outlier districts are in Northern California or in the Bay Area relative to 

the total number of districts in these areas. Instead, positive outlier districts are disproportionately 
concentrated in Southern California. One reason for the higher concentration in Southern California 
is that this region includes more districts with over 2,000 students. 


Figure 5 
Map of 54 Positive Outlier Districts 
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Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 
(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. (n.d.). Education 
demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 
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Part Il. Predictors of Student Achievement 


Methodology 


In Part II of our analysis, we use the same two-level hierarchical linear model used in Part I to 
identify the factors most strongly associated with African American, Hispanic, and White student 
achievement. As in Part I, we measure achievement of students using 3 years of CAASPP scores 
averaged across subjects. (See Appendix A for details about the model.) To improve the reliability 
of our estimates, we only include districts with at least 200 African American or Hispanic students 
and 200 White students, so our sample includes 435 California districts. Using publicly available 
data, we develop three models to predict how well a district’s students achieve on the first 3 years of 
CAASPP, controlling for the SES conditions in the district. Table 2 includes the descriptive statistics 
for each variable in our analysis. When we run our analysis, we center the variables in Table 2 so 
they have a mean of zero and the same standard deviation that is reported in Table 2. Selected 
results from our analyses are included in Table 3 and discussed in detail below. See Appendix A for 
the full results from our analysis and more information about our methodology. 


Table 2 
Descriptive Statistics 


Standard 


NV EVat-le) (=) Deviation Data Source 


California Department of Education 


African American & Hispanic Score -0.219 0.273 CAASPP Results for 2015-2017 


California Department of Education 


White Score 0.220 0.362 CAASPP Results for 2015-2017 


SES: Percent of Parents With National Center for Education 


Bachelor’ Ab 0.229 0.157 Statistics Education Demographic and 
SEN BL ae Nee Geographic Estimates for 2006-2013 

: National Center for Education 
SES over Rate ol enw a 0.189 0.142 Statistics Education Demographic and 


HOLES CAIN SE OIG Geographic Estimates for 2006-2013 


National Center for Education 
SES: Percent Using SNAP Benefits 0.153 0.118 Statistics Education Demographic and 
Geographic Estimates for 2006-2013 


National Center for Education 
0.324 0.124 Statistics Education Demographic and 
Geographic Estimates for 2006-2013 


SES: Percent of Households 
Headed by a Single Mother 


National Center for Education 
SES: Unemployment Rate 0.076 0.032 Statistics Education Demographic and 
Geographic Estimates for 2006-2013 


National Center for Education 


SES: Median Income ($) 69,650 337139 Statistics Education Demographic and 
Geographic Estimates for 2006-2013 


SES: Percent Economically California Department of Education 


0.525 0.277 


Disadvantaged Enrollment by School for 2015-2017 
National Center for Education 
# of Students (Logged) 8.818 1.040 Statistics Common Core of Data 


for 2015 
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Standard 
NVEVat-le) (=) Deviation Data Source 
National Center for Education 


Student-Teacher Ratio Statistics Common Core of Data for 
2015 


California Commission on Teacher 


A , 
Teacher oalaty Byes co adlusied Credentialing Teacher Supply Data 


for Cost of Living (Logged) for 2016 


California Department of Education 


Avg. Years Teaching Experience in District Staff Demographic Data for 2016 


Percent Teachers With Intern Credentials, Celuduley Celle oluela Peele 
Credentialing Teacher Supply Data 


Temporary or Short-Term Permits, or Waivers for 2016 


United States Census Bureau Annual 
Total Per-Pupil Expenditures (Logged) Survey of School System Finances 
for 2015 


United States Census Bureau Annual 
Percent Spending on Instruction Survey of School System Finances 
for 2015 


Model 1: District characteristics 


Our first model examines the relationship between student achievement and a district’s size and 
student characteristics. In this model, we include the six SES variables described above that account 
for the SES characteristics for families of a given racial/ethnic group in each district. As in Part I, 

we also include a measure of the proportion of socioeconomically disadvantaged students by race 
who were tested in each school district, as reported by the California Department of Education. 

In addition, we include a measure of district size in the model, with a variable representing the 
number of students enrolled in each district (in log form) as reported by the federal National Center 
for Education Statistics Common Core of Data.° 


Model 2: District and teacher characteristics 


Model 2 adds to the variables in Model 1 by including measures of teaching characteristics in each 
district, such as the number of students per teacher, as well as teachers’ compensation, experience, 
and training. 


As they make hiring decisions, districts balance the quantity and quality of teachers. Quality is 
often influenced by teachers’ education, training, and experience,® which are recognized in the 
compensation system; thus, hiring a greater quality of teachers with a given level of dollars may 
mean hiring a smaller quantity of teachers. To examine teacher quantity, we include each district’s 
average student-to-teacher ratio as reported by the federal National Center for Education Statistics 
Common Core of Data. 


To estimate compensation, we include a measure of each district’s average salary for teachers with 
a bachelor’s degree and 60 semester units or hours of additional education (in log form). This allows 
us to reflect districts’ salary schedules in a comparable way, benchmarking salaries for similarly 
educated teachers across districts, without confounding our wage estimate by the very different 
average years of teacher experience across districts. The BA + 60 benchmark captures the salary 
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many mid-career teachers would earn. In California, most teachers are trained in postbaccalaureate 
programs, so they come into teaching with a BA + 30 units (or 45 if they have earned a master’s 
degree), and they then incrementally acquire additional credits through professional development 
experiences to move up the salary scale in the succeeding years. These salary data are voluntarily 
reported to the California Department of Education, so we do not have salary data for all California 
districts. Consequently, we use the statewide mean salary at this education level for districts with 
missing salary values. We also include a dummy variable indicating whether or not districts publicly 
reported their salary data. For districts with salary data, we adjust salaries for the cost of living in 
each district using the 2013 comparable wage index.’ 


In addition, we include a variable reported by the California Department of Education in our model 
to measure the average years of teaching experience within the district for teachers in each district. 


To estimate teachers’ training, we include a variable that measures the percent of teachers with 
substandard credentials (i.e., intern credentials, temporary or short-term permits, or waivers) in 
each district as reported by California’s Commission on Teacher Credentialing for the 2015-16 
school year. This measure indicates the percentage of teachers who have not yet completed a 
teacher preparation program nor met the standards for a full teaching credential in California. 
During the ongoing teacher shortages in California, the Teacher Credentialing Commission has 
issued a growing number of such authorizations. In 2015-16, more than 10,000 substandard 
credentials were issued, about half of all credentials issued in that year.® 


Model 3: District, teacher, and financial characteristics 


Model 3 adds to the variables included in Models 1 and 2 by including measures of financial 
allocations in the district. Using the most recent federally available school finance data reported 

by the United States Census Bureau from the 2014-15 school year (the most recent year available), 
we create two measures of district resource allocations. The first measure is a variable of the 
average per-pupil expenditures in a district (in log form). We also include the average percentage of 
expenditures districts allocate toward instruction, which includes the amounts districts spend on 
teachers’ salaries and their non-retirement benefits, as well as instructional books and resources. 


Table 3 
Results for Correlates of California Student Achievement 


African African African 

American and American and American and 

Hispanic Score Hispanic Score Hispanic Score 
SES: Percent of Parents 0.847*** 0.916*** 0.866*** 0.946*** 0.876*** 0.907*** 
WHRBachelarsorAbove | 4s) (0.073) 0.112) (0.073) (0.114) 0.080) 
SES: Poverty Rate of -0.148 0.081 -0.122 0.079 -0.108 0.073 
Head of Household With 
5- to 17-Year-Olds (0.086) (0.106) 0.081) (0.103) (0.082) 0.1.03) 
SES: Percent Using -0.047 -0.106 -0.039 -0.048 -0.051 -0.068 
Sher Benefits (0.095) (0.127) 0.091) (0.124) (0.091) 0.125) 
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NY Coe (=) lie Model 2 NY Coe (=) Ic} 


African African African 
American and American and American and 
Hispanic Score Hispanic Score Hispanic Score 


SES: Percent of -0.039 -0.140 -0.021 -0.119 -0.017 -0.123 
Households With Single 
Mother Head (0.072) (0.081) (0.069) (0.079) (0.068) (0.079) 


SES: Unemployment -0.162 -0.054 0.031 -0.103 0.013 -0.186 


Rate 


(0.237) (0.351) (0.229) (0.348) (0.228) (0.347) 


0.014 0.003 0.020 0.002 0.017 0.002 


SES: Median Income 
(0.030) (0.033) (0.029) (0.032) (0.029) (0.032) 


SES: Percent -0.539*** -0.788*** -0.506*** -0.783*** -0.493*** -0.778*** 
Economically 
Disadvantaged (0.062) (0.059) (0.060) (0.057) (0.061) (0.061) 


0.002 0.009 -0.014 -0.003 -0.013 -0.003 
# of Students (Logged) 


(0.008) (0.007) (0.008) (0.007) (0.008) (0.007) 


0.005 0.004 0.003 0.005 


Student-Teacher Ratio 
(0.003) (0.003) (0.004) (0.004) 


Teacher Salary at BA + 0.033 0.053 0.036 0.042 
60, Adjusted for Cost of 
Living (Logged) (0.039) (0.036) (0.039) (0.037) 


Average Years Teaching 0.008* 0.006 0.008* 0.006 


Experience in District 


(0.004) (0.003) (0.004) (0.003) 


SORT OIE I -0.919*** -0.727*** -0.884*** -0.677** 
Intern Credentials, 


Temporary or Short-Term 
Permits, or Waivers 


(0.224) (0.220) (0.224) (0.220) 


Total Per-Pupil -0.075 0.076 


Expenditures (Logged) 


(0.069) (0.068) 


Percent Spending on 0.163 0.285* 


Instruction 


(0.121) (0.114) 


District Observations 


R-Squared 


Note: Standard errors in parentheses: “*“p < 0.01, ““p < 0.05, *p < 0.1. 


Data sources: California Commission on Teacher Credentialing. (n.d.). Teacher supply: Credentials. https://www.ctc.ca.gov/ 
commission/reports/data/edu-supl-landing (accessed 01/05/18); California Department of Education. (n.d.). California 
Assessment of Student Performance and Progress (CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); 
California Department of Education. (n.d.). Enrollment by school. https://www.cde.ca.gov/ds/sd/sd/filesenr.asp (accessed 
12/29/17); California Department of Education. (n.d.). Staff demographic data. https://www.cde.ca.gov/ds/sd/df/filesstaff- 
demo.asp (accessed 01/05/18); National Center for Education Statistics. (n.d.). Common Core of Data. https://nces.ed.gov/ 
ccd (accessed 01/05/18); National Center for Education Statistics. (n.d.). Education demographic and geographic estimates. 
https://nces.ed.gov/programs/edge (accessed 01/05/18); United States Census Bureau. (n.d.). Annual survey of school 
system finances. https://www.census.gov/programs-surveys/school-finances.htm| (accessed 01/05/18). 
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Results 


Model 1: District characteristics 


Our first model is informed by research showing that district socioeconomic characteristics are 
strongly associated with student achievement.’ For example, beginning in 1966, the Coleman 
Report famously highlighted the significant relationship between family socioeconomic status 

and student achievement. Countless studies have confirmed that the socioeconomic status of 
students’ parents is one of the strongest predictors of their educational achievement.” In addition, 
numerous studies have found that schools with smaller student enrollments are associated with 
improved student achievement, and some have suggested that districts with smaller student 
enrollments have stronger performance, although the evidence on this score is more mixed.'! Some 
studies have found that smaller schools and districts are especially beneficial for students from 
low-income families.” 


The results of Model 1 in Table 3 show that the average socioeconomic status of families, as 
measured by their educational attainment, as well as the district’s percent of students from 
low-income families, is indeed significantly associated with student achievement. In addition, we 
do not find a significant relationship between the number of students enrolled in a district and 
student achievement. 


Model 2: District and teacher characteristics 


The percent of teachers 
Our analyses confirm the widespread P 


finding that teachers play an important role holding substandard 

in contributing to student achievement. credentials is significa ntly and 
Teachers are often considered to be the most 
important within-school contributors to 
student achievement.!? We examine several student achievement. 

characteristics suggested by other research 

to be predictive of teacher effectiveness: 

student-teacher ratio, '* the level of the 

teacher salary scale,’ average teaching experience,'® and full certification as a measure of teacher 
qualifications.'’ These variables are also at least modestly correlated with each other. For example, 
teachers teaching on substandard credentials (i.e., intern, permit, or waiver credentials), are 

often more likely to be found in relatively lower paying districts, and they tend to be the least 
experienced and effective.'® 


negatively associated with 


In Model 2, we find that the extent of preparation as reflected by teacher certification status has 

a strong association with average achievement for all students. After controlling for salaries and 
experience, the percent of teachers holding substandard credentials is significantly and negatively 
associated with student achievement. In these districts, for every 10% increase in the percent of 
teachers working on emergency permits, waivers, or intern credentials, the average achievement for 
students of color is lower, on average, by almost 0.10 standard deviations. For White students, every 
10% increase in the percent of teachers teaching on substandard credentials is associated with 
achievement that is nearly .07 standard deviations lower. 
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In addition, we find that for African American and Hispanic students, average teacher experience is 
positively associated with student achievement. Although we do not find that teacher salary levels 
have a direct association with achievement in this model, salary levels are often associated with the 
qualifications of teachers, which are, in turn, associated with student achievement.'” 


When interpreting the results from this model, we note that higher percentages of teachers with 
substandard credentials may be a symptom of districts with a weaker labor market or with weaker 
teaching and learning conditions. For example, districts in rural areas with less proximity to schools 
of education and many amenities struggle to recruit and retain teachers, as do under-resourced 
communities, which often struggle as well to retain strong principals and provide sufficient 
teaching resources. These places that are difficult for teachers to work in and students to learn in 
may feel they need to hire more teachers on substandard credentials because relatively few teachers 
want to work in the district.”° It is also true that districts are differentially focused on recruiting and 
retaining staff, and that some spend more of their money and effort to recruit and retain a strong 
teaching staff than others.?! 


Whatever the sources of substandard credentials, this finding highlights the importance of teacher 
characteristics as indicators of both the teaching and learning conditions within a district and as 
correlates of student achievement. 


Model 3: District, teacher, and financial characteristics 


Model 3 adds to the variables included in Models 1 and 2 by including measures of financial 
allocations in the district. In some other studies, higher per-pupil spending has been found to 

be associated with improved student achievement.” In addition, increased district expenditures 
allocated toward instruction’ and investments in teacher quality** have been found to be especially 
effective at raising student achievement. 


In our analysis, after controlling for student-teacher ratios, teacher salary levels, and teacher 
qualifications, total school spending does not have a statistically significant association with 
student achievement. This is not surprising, as these variables capture the major elements of 
total expenditures. Beyond overall spending, the percent of spending on instruction is moderately 
associated with the achievement of White students. 


Interestingly, the significant association between underprepared teachers and the achievement of 
both students of color and White students remains in this model, as does the association between 

teacher experience and achievement for students of color. This confirms the well-documented role 
of teachers in supporting the achievement of all students, especially students of color. 
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Conclusions 


These initial analyses indicate that a 

substantial number of districts in California are Districts that have been able 
outperforming expectations for their students’ to avoid the effects of the 
achievement on the state’s new, more rigorous 
assessments measuring deeper learning, and 
that these positive outliers are visible in large by recruiting and retaining fully 


and small districts in urban and rural settings prepared teachers are much more 
throughout the state. 


widespread teacher shortages 


likely to produce strong student 
The analyses also show that, aside from 
socioeconomic status, a major predictor of 
student achievement is the preparedness of 
teachers. In our analyses, we used credentialing for White students. 
and experience as proxies for this knowledge 
and skill base (i.e., whether teachers hold a full 
credential, rather than an intern credential, 
temporary or short-term permit, or waiver for their teaching position). We recognize that a 
concentration of such teachers is also a sign of difficulty recruiting and retaining staff, which may 
signal broader differentials in teaching and learning conditions as well as teacher quality. Districts 
that have been able to avoid the effects of the widespread teacher shortages by recruiting and 
retaining fully prepared teachers are much more likely to produce strong student achievement for 
African American and Hispanic students as well as for White students. 


achievement for African American 
and Hispanic students as well as 


To shed light on the practices that support student achievement, the Learning Policy Institute 
investigated a set of positive outlier districts during the 2017-18 school year to better understand 
how they support student achievement, especially the achievement of students of color. This 
study used in-depth interviews and observations in seven positive outlier school districts from 
across the state—Chula Vista Elementary, Clovis Unified, Gridley Unified, Hawthorne Unified, 
Long Beach Unified, San Diego Unified, and Sanger Unified—to identify factors that contribute to 
student success as reflected in state assessments and other outcome data, such as low suspension 
rates and high graduation rates. The case studies are published as a separate report identifying the 
instructional and other policies and practices found in common across these positive outliers. 
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Appendix A: Methodology 


Constructing District-Level Socioeconomic Variables 


To calculate district-level socioeconomic status (SES) in both Part I and Part II of our analysis, 

we include a measure of the number of socioeconomically disadvantaged students tested by race 

in each school district, as reported by the California Department of Education. To be considered 
socioeconomically disadvantaged, students must meet at least one of the two criteria: neither of their 
parents received a high school diploma, or they are eligible to receive free or reduced-price lunch. 


We also include another six district SES measures constructed from publicly available American 
Community Survey’s (ACS) Education Demographic and Geographic Estimates (EDGE) data. We use 
three data sets to construct the measures: (1) socioeconomic characteristics of the total population 
residing in each district, averaged over 2006-10; (2) socioeconomic characteristics of the families 
living in the district who have children enrolled in public school, averaged over 2006-10; and 

(3) socioeconomic characteristics of the total population residing in each district, averaged 

over 2009-13. 


Ideally, we would like measures of the socioeconomic characteristics of families whose children 
were enrolled in public schools in each district in 2015-17, as those are the years that correspond to 
the populations taking the California Assessment of Student Performance and Progress (CAASPP) 
tests in our data. However, the ACS only provides small area EDGE estimates for 5-year windows, the 
most recent of which is 2009-13. Moreover, the only ACS tabulations in 2009-13 describe the total 
population in each district, rather than the population of families with children in public school, 
which is the population of interest. That population only has ACS EDGE data from 2006-10. 


In order to construct estimates of the socioeconomic characteristics of families with children in 
public schools in 2009-13 (the closest years we can get to the 2015-17 CAASPP testing years), we 
do the following: 


1. We first estimate the relationship between the SES measures in the total population and of the 
children in a district in the 2006-10 sample with the following regression: 


SES Childg®*® = [SES Total9®?"|B + eq 


where SES Child9®"” is one of the six child socioeconomic variables measured in 2006-10, 
[s ES Total?°"| is a vector of all six socioeconomic variables of the total population in the 
district, measured in 2006-10. We then capture the values of B and é4. 


2. Next, we predict the socioeconomic characteristics of the families with children enrolled in public 
school in 2009-13, using the following equation: 
SES Childj?™? = [SES Total#?'?|B + éq 


where SES C mid? is the predicted value of one of the six child variables in 2009-13, 
[s ES Total??"?| is a vector of the six socioeconomic variables of the total population in 2009-13, 
and B and é, are the fitted coefficients and residuals from the regression above. 
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With this method, we assume that the relationship between the SES of the total population and 
the population of public school children in a district, as well as the deviation from that predicted 
relationship, is the same between the 2006-10 and 2009-13 samples. 


Model and Parameters 


We use a two-level, precision-weighted hierarchical linear model to identify the districts that are 
performing better on CAASPP than one would predict based on the SES of the families each district 
serves. We use a multilevel model because we observe average test scores in multiple grades, years, 
and subjects in each district; thus, the model accounts for the nesting of grade-year-subject cells 
within districts. The precision weighting gives more weight in the estimation to cells whose mean 
test scores are more precisely estimated (that is, it gives more weight to observations based on 
larger numbers of students). The model can be written as follows: 


Ya = Gat Gq (grade -—G)+ ega t+ Ta 
Goa = Boo + BorSESa + Uoa 
A1aq = Biot Pui SESa + Ug 


Y, 


ga~N; [0,024]; ega~N[0, 07]; (°°) ~MVN[O, t?]. 


gd 1d 


In this model, 


A 


° Yoa is the estimated average English language arts or math score for a particular district, 
grade, year, and subgroup, standardized with the state mean and standard deviation. 


® Qpoq is a random district-specific intercept that indicates the average test scores in district 
d and grade G (where the linear grade term is centered on grade G). For districts serving 
grades 3-8, we center the grade at 5.5. For districts serving grades 3-8 and 11, we center 
the grade at 7. In this way, the intercept estimates the district’s performance at its “average” 
grade. Qq has a mean of Bog and a variance of To, both of which must be estimated. 


° Q,qis a district-specific grade slope; it indicates the linear trends in average test scores 
across grades within district d. aq has a mean of Bj, and a variance of T7,, both of which 
must be estimated. 


° Tgq is the sampling error of oe We assume it is normally distributed with mean zero and 
variance woa=l/Ngw where Noq is the number of students tested. 


*® gq is the level 1 error (the deviation of the true value of Y,q from the linear trend in Y 


across grades in district d), which we also assume to be normally distributed with a mean of 
0 and a variance, 7%, to be estimated. 
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Parameters 


The parameters of interest are as follows: 


We rank districts based on their value of ug,q, the empirical Bayes “shrunken” estimate of Ugg. This 
tells us the difference between a district’s actual average performance and what one would predict 
the performance of the district to be based on its socioeconomic characteristics. In all cases, a larger 
positive residual means a district performed better than expected for students of a given racial/ 
ethnic group controlling for their families’ socioeconomic status. (A larger negative residual means 
a district performed worse than expected.) We also provide the standard error of this residual and 
calculate the 95% confidence interval. 


We also include a fitted value of the average district performance. We calculate this value as follows: 


This is a district’s predicted average performance of the district based solely on its socioeconomic 
variables. We add this estimate so that it is clear how well a district was predicted to perform, and 
one can easily compare that to how the district actually performed. 


Finally, we include an empirical Bayes “shrunken” estimate of the average achievement in the 
district, denoted @q = Boo + ip. This parameter is estimated from a two-level model that is 
identical to the one above, but without the SES measures. This provides a “shrunken” estimate of 
the average performance in each district (shrunken so that small districts with imprecise estimates 
get discounted). 


Reliability of estimates 


We include reliability estimates in 2015 for the intercepts @gq. The rankings are only informative 

if we can reliably calculate the parameters of interest. As shown in Table A1, the reliabilities of the 
intercept from models that include socioeconomic variables and from which we obtain the residuals 
range from a high of 0.911 to a low of 0.766. The reliabilities of the district means that come from 
models that do not include SES range from a high of 0.978 to a low of 0.876. The vast majority of 
parameters are therefore estimated with a high reliability. 
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Table A1 
Reliability of Estimates 


Reliability of Average District Reliability of Average District 
Subgroup Subject Performance (SES adjusted models) Performance (unadjusted models) 


All 


African 
American 


Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 
(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. (n.d.). Education 
demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 


Analysis 


For our Part II analysis of achievement predictors, we add measures of district, teacher, and 
financial characteristics to the hierarchical linear models described above. We include these district- 
level measures on the second level in our model. For these analyses, we center all of the variables, 
except the dummy variables. To calculate the socioeconomic variables for the African American 

and Hispanic combined group, we estimate the weighted average for each socioeconomic variable 
by grade and year and then take the average of these estimates to calculate the district-level 
socioeconomic variable. We include our full results in Table A2 below. 


The full results of the analysis include three key additions to what is noted in the body of this 
report. First, the full results include the intercept of the model. The intercept can be thought of as 
the average test score in the average grade (in this case approximately grade 6 because we centered 
grade) in the “average” district. 


Second, the results include grade-level slopes of the variables in our analysis. The coefficient on 
“Grade” can be thought of as the average change in test scores, per grade, relative to the statewide 
average. Because the test scores are standardized within each grade and year and subject, a value of 
0 represents average growth; positive values indicate above-average improvement rates; negative 
values indicate below-average improvement. The coefficients on the interaction terms with grade 
(covariate_x_grade terms) can be interpreted as the relationship between the covariate and the 

rate of average test score improvement across grades. For example, the negative coefficient on 

# Students_x_Grade indicates that districts with larger student enrollments have lower average test 
score improvement per grade than do smaller districts. 


Third, we include several variables to examine potential confounding relationships. To address the 
issue of districts with large enrollments biasing our results, we include a dummy variable for Los 
Angeles Unified School District (in addition to using the logged variable for student enrollment). 
We also include the percentage of students receiving special education services in our models. 

For the teacher salary data, we include a dummy variable for whether a district was missing 

salary data, because not all districts report these data. We find that there is not a relationship 
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between the districts missing salary data and White student achievement, and only a modest 
association between districts missing salary data and African American and Hispanic achievement. 
This suggests that our approach to imputing missing values with average salary data does not 
significantly bias our results. 


Table A2 


Full Results of Hierarchical Linear Model Analysis of Correlates of 
Student Achievement 


NY Coe (=) Iie Model 2 NY Coe (=) fac} 
African African African 
American and American and American and 
Hispanic Score Hispanic Score Hispanic Score 

-0.217*** 0.207*** -0.213*** 0.208*** -0.212*** 0.210*** 
Intercept 

(0.007) (0.006) (0.007) (0.006) 0.007) (0.006) 
SES: Percent of Parents 0.847*** fe jallsysess OS 66g 0.946*** OB16**=* ONO eas 
LAIR CLISTIS yeh /aee || gy aly (0.073) (0.112) (0.073) 0.114) (0.080) 
SES: Poverty Rate of -0.148 0.081 -0.122 0.079 -0.108 0.073 
Head of Household With 
5- to 17-Year-Olds (0.086) 0.106) (0.081) (0.103) 0.082) (0.103) 
SES: Percent Using -0.047 -0.106 -0.039 -0.048 -0.051 -0.068 
a eenet ie (0.095) 0.127) (0.091) (0.124) 0.091) (0.125) 
SES: Percent of -0.039 -0.140 -0.021 -0.119 -0.017 -0.123 
Households With Single 
Mother Head (0.072) 0.081) (0.069) (0.079) 0.068) (0.079) 
SES: Unemployment -0.162 -0.054 0.031 -0.103 0.013 -0.186 
Bete (0.237) (0.351) (0.229) (0.348) 0.228) (0.347) 

0.014 0.003 0.020 0.002 0.017 0.002 
SES: Median Income 

(0.030) (0.033) (0.029) (0.032) 0.029) (0.032) 
SES: Percent AOL Sishejereres =“0:1/88"** FOIDOG ns + SONRS Sues -0.493*** AO) 7 Theos 
Economically 
Disadvantaged (0.062) (0.059) (0.060) (0.057) 0.061) (0.061) 

0.002 0.009 -0.014 -0.003 -0.013 -0.003 
# of Students (Logged) 

(0.008) (0.007) (0.008) (0.007) 0.008) (0.007) 

0.013 -0.088 0.069 -0.042 0.076 -0.041 
Dummy for LAUSD 

(0.146) (0.133) (0.139) (0.128) 0.139) (0.127) 

0.005 0.004 0.003 0.005 
Student-Teacher Ratio 
(0.003) (0.003) 0.004) (0.004) 

Percent Students With -0.848** -0.525 -0.822** -0.659* 
Seca elll Bees (0.284) (0.270) 0.291) (0.276) 
Teacher Salary at BA + 0.033 0.053 0.036 0.042 
60, Adjusted for Cost of 
Living (Logged) (0.039) (0.036) 0.039) (0.037) 
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NY Coe (=) fie 


African 


American and 


Model 2 


African 
American and 


NY Coe (=) Ic} 


African 
American and 


Hispanic Score 


Hispanic Score 


Hispanic Score 


-0.058* -0.018 -0.078* -0.048 
Missing Salary Data 
(0.028) (0.026) (0.031) (0.029) 
Avg. Years Teaching 0.008* 0.006 0.008* 0.006 
Experlenge In Distitt (0.004) (0.003) (0.004) (0.003) 
PERSE SITS ae) -0.919*** -0.727*** -0.884*** -0.677** 
Intern Credentials, 
Temporary or Short-Term 
Permits, or Waivers (0.224) (0.220) (0.224) (0.220) 
Total Per-Pupil ae cd 
Expenditures (Logged) 0.069) (0.068) 
Percent Spending on ues Tes 
Instruction 0.121) (0.114) 
0.001 -0.005*** 0.001 -0.005*** 0.001 -0.004** 
Grade 
(0.001) (0.001) (0.001) (0.001) 0.001) (0.001) 
-0.015 -0.019 -0.012 -0.011 -0.020 -0.016 
BA_x_Grade 
(0.021) (0.016) (0.021) (0.016) (0.022) (0.018) 
0.030 0.011 0.034* 0.011 0.034* 0.008 
Poverty_*x_Grade 
(0.016) 0.025) (0.016) (0.025) 0.016) (0.025) 
-0.011 0.002 -0.011 0.002 -0.009 0.002 
SNAP_x_Grade 
(0.018) 0.029) (0.018) (0.030) 0.018) (0.030) 
-0.009 -0.031 -0.010 -0.028 -0.010 -0.029 
Single_x_Grade 
(0.013) 0.019) (0.013) (0.019) 0.013) (0.019) 
0.042 0.105 0.056 0.130 0.051 0.119 
Unemployment_x_Grade 
(0.043) (0.079) (0.044) (0.080) (0.044) (0.081) 
, -0.001 0.010 -0.001 0.009 0.000 0.009 
Median 
Income_*_Grade (0.006) 0.008) (0.006) (0.009) 0.006) (0.009) 
0.008 0.027 0.011 0.026 0.009 0.027 
Percent Poor_x_Grade 
(0.011) 0.014) (0.011) (0.014) 0.012) (0.015) 
0.022 -0.020 0.027 -0.018 0.026 -0.018 
LAUSD_*x_Grade 
(0.023) 0.024) (0.023) (0.024) 0.023) (0.024) 
-0.004** -0.004* -0.006*** -0.005** -0.006*** -0.005** 
# Students_x_Grade 
(0.001) (0.001) (0.001) (0.002) (0.001) (0.002) 
Student-Teacher 0.001 0.001 0.001 0.001 
Ratlo_x_Grade (0.001) (0.001) (0.001) (0.001) 
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NY Coe (=) Iie Model 2 NY Coe (=) fac} 


African African African 
American and American and American and 
Hispanic Score Hispanic Score Hispanic Score 


Percent Special 
Needs_x_Grade 


Teacher Salary_x_Grade 


Missing Salary_x_Grade 


Teaching 
Experience_x_Grade 


Teacher 
Credentials_x_Grade 


Per-Pupil 
Expenditures_x_Grade 


Spending 
Instruction_x_Grade 


R-Squared 


Note: Standard errors in parentheses: ***p < 0.01, “*p < 0.05, “p< 0.1. 


Data sources: California Commission on Teacher Credentialing. (n.d.). Teacher supply: Credentials. https://www.ctc.ca.gov/ 
commission/reports/data/edu-supl-landing (accessed 01/05/18); California Department of Education. (n.d.). California 
Assessment of Student Performance and Progress (CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); 
California Department of Education. (n.d.). Enrollment by school. https://www.cde.ca.gov/ds/sd/sd/filesenr.asp (accessed 
12/29/17); California Department of Education. (n.d.). Staff demographic data. https://www.cde.ca.gov/ds/sd/df/filesstaff- 
demo.asp (accessed 01/05/18); National Center for Education Statistics. (n.d.). Common Core of Data. https://nces.ed.gov/ 
ccd (accessed 01/05/18); National Center for Education Statistics. (n.d.). Education demographic and geographic estimates. 
https://nces.ed.gov/programs/edge (accessed 01/15/18); United States Census Bureau. (n.d.). Annual survey of school 
system finances. https://www.census.gov/programs-surveys/school-finances.html (accessed 01/15/18). 
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Appendix B: List of Positive Outlier Districts for 
African American and Hispanic Students 


The two tables below list positive outlier districts, which are ranked according to the size of 

their residual for African American and Hispanic students, respectively. Districts with the largest 
residual—where African American and Hispanic students achieve much higher than predicted—are 
listed first. The district residuals are averaged by subjects, grades, and across years (i.e., 2015-17). 


Table B1 
List of Positive Outlier Districts for African American Students 


As Presented in Figure 3 


Chula Vista Elementary 0.213 0.233 
Perris Elementary 0.199 0.048 
Etiwanda Elementary 0.180 0.174 
Alvord Unified ORAS 0.111 
Santa Monica-Malibu Unified 0.173 0.018 
ABC Unified 0.170 0.230 
Lemon Grove 0.167 0.181 
Redlands Unified 0.163 0.106 
San Diego Unified 0.160 0.142 
San Marcos Unified 0.154 0.200 
Greenfield Union 0.146 0.110 
Lawndale Elementary 0.144 0.192 
Hawthorne 0.142 0.119 
Downey Unified 0.140 0.093 
Long Beach Unified 0.119 0.117 
San Bernardino City Unified OMA 0.092 
Tustin Unified 0.109 0.008 
Upland Unified 0.106 0.031 
Desert Sands Unified 0.106 0.112 
Corona-Norco Unified 0.085 0.059 
Culver City Unified 0.080 0.087 
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District Name African American Student Residual White Student Residual 


Visalia Unified 


William S. Hart Union High 


Val Verde Unified 


La Mesa-Spring Valley 


Center Joint Unified 


Bellflower Unified 


Twin Rivers Unified 


Merced City Elementary 


Sacramento City Unified 


Vista Unified 


Central Elementary 


Dry Creek Joint Elementary 


Riverside Unified 


Torrance Unified 


Palm Springs Unified 


Bakersfield City 


Apple Valley Unified 


Fremont Unified 


Oceanside Unified 


Grossmont Union High 


Elk Grove Unified 


Anaheim Union High 


Brentwood Union Elementary 


Murrieta Valley Unified 


Cucamonga Elementary 


Fresno Unified 


Alta Loma Elementary 


Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 
(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. (n.d.). Education 
demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 
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Table B2 
List of Positive Outlier Districts for Hispanic Students 


As Presented in Figure 4 


District Name Hispanic Student Residual White Student Residual 


Newhall 0.354 0.300 
Hawthorne 0.326 0.119 
Winton 0.310 0.384 
Palo Verde Union Elementary 0.307 0.061 
La Canada Unified 0.298 0.084 
Little Lake City Elementary 0.286 0.274 
Coast Unified 0.283 0.070 
Magnolia Elementary 0.280 0.354 
Carmel Unified 0.271 0.182 
Gridley Unified 0.255 0.122 
Solvang Elementary 0.254 0.131 
Clovis Unified 0.249 0.180 
Kings Canyon Joint Unified 0.240 0.163 
Lawndale Elementary 0.236 0.192 
Sulphur Springs Union 0.233 0.234 
Holtville Unified 0.228 0.122 
Los Alamitos Unified 0.223 0.179 
Solana Beach Elementary 0.221 0.058 
ABC Unified 0.220 0.230 
Atwater Elementary 0.218 0.243 
Sanger Unified 0.214 0.257 
Kingsburg Elementary Charter 0.213 0.050 
Wright Elementary 0.212 0.032 
Chula Vista Elementary 0.204 0.233 
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District Name Hispanic Student Residual White Student Residual 


Sundale Union Elementary 0.203 0.334 
Caruthers Unified 0.197 0.093 
Bassett Unified 0.196 0.073 
Downey Unified 0.195 0.093 
Nuview Union 0.194 0.141 
Kerman Unified 0.191 ONiar, 
Fallbrook Union Elementary 0.190 0.291 
Placentia-Yorba Linda Unified 0.184 OMS 
Live Oak Unified 0.183 0.190 
Encinitas Union Elementary Oxisal 0.048 
Rocklin Unified 0.175 0.083 
Etiwanda Elementary 0.174 0.174 
East Whittier City Elementary 0.173 0.146 
Greenfield Union 0.167 0.110 
Val Verde Unified 0.166 0.167 
Del Mar Union Elementary 0.166 0.123 
Centralia Elementary 0.165 0.262 
North County Joint Union Elementary 0.164 0.141 
South Bay Union 0.157 0.225 
Duarte Unified Onli 0.031 
Mother Lode Union Elementary 0.157 0.053 
Carlsbad Unified OMai4 0.148 
Riverdale Joint Unified 0.156 0.007 
Perris Elementary 0.154 0.048 
McCabe Union Elementary 0.153 0.117 
Firebaugh-Las Deltas Unified 0.150 0.016 
Ocean View 0.149 0.092 
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District Name Hispanic Student Residual White Student Residual 


Delhi Unified 0.148 0.084 
San Marcos Unified 0.148 0.200 
West Covina Unified 0.147 0.154 
Mesa Union Elementary 0.145 0.041 
Central Union High 0.142 0.003 
Hanford Elementary 0.141 0.221 
EI Monte City 0.139 0.398 
Antelope Valley Union High 0.138 0.069 
Fruitvale Elementary OMSiG 0.132 
Carpinteria Unified 0.133 0.060 
Bellflower Unified OMis2. Onis: 
Saint Helena Unified 0.131 0.053 
Weaver Union OHIZ9 0.100 
Whittier Union High 0.128 0.157 
Redlands Unified 0.124 0.106 
National Elementary 0.122 0.198 
Monrovia Unified 0.122 0.161 
Glendora Unified 0.121 0.140 
Oak Valley Union Elementary 0.119 OHs5 
South Pasadena Unified 0.116 0.148 
Brea-Olinda Unified 0.115 0.032 
Savanna Elementary 0.114 0.149 
Desert Sands Unified 0.113 0.112 
Corona-Norco Unified 0.113 0.059 
Covina-Valley Unified Oud 0.123 
Fowler Unified 0.111 0.229 
Garden Grove Unified 0.110 0.120 
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District Name Hispanic Student Residual White Student Residual 


Huntington Beach City Elementary 0.109 0.165 
Central Union Elementary 0.109 0.254 
Ceres Unified 0.107 0.064 
Lakeport Unified 0.106 0.033 
Burlingame Elementary 0.105 0.163 
Waterford Unified 0.104 0.135 
Fountain Valley Elementary 0.102 0.093 
Long Beach Unified 0.101 OMMi7, 
Selma Unified 0.101 0.162 
Snowline Joint Unified 0.099 Ome 
Mountain View Whisman 0.098 0.288 
Hart-Ransom Union Elementary 0.097 0.016 
William S. Hart Union High 0.096 0.075 
Los Altos Elementary 0.095 0.083 
Pioneer Union Elementary 0.094 0.060 
Imperial Unified 0.092 0.148 
Cypress Elementary 0.092 0.042 
Lemon Grove 0.092 0.181 
Keyes Union 0.091 0.097 
Central Elementary 0.090 0.083 
Bonita Unified 0.090 0.077 
Westminster 0.088 0.250 
Glendale Unified 0.087 0.135 
Lemoore Union Elementary 0.087 0.056 
San Bernardino City Unified 0.087 0.092 
Porterville Unified 0.087 0.091 
Chino Valley Unified 0.086 0.054 
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District Name Hispanic Student Residual White Student Residual 


Murrieta Valley Unified 0.085 0.035 
Red Bluff Union Elementary 0.083 0.143 
Tustin Unified 0.081 0.008 
Santee 0.074 0.122 
Gilroy Unified 0.072 0.028 
Moreland 0.072 0.207 
Visalia Unified 0.070 0.115 
El Centro Elementary 0.068 0.049 
Riverside Unified 0.066 0.048 
Tahoe-Truckee Unified 0.065 0.020 
Palo Alto Unified 0.065 0.052 
Colusa Unified 0.065 0.053 
Rowland Unified 0.064 0.190 
Lowell Joint 0.063 0.145 
Alvord Unified 0.063 ONE 
Galt Joint Union Elementary 0.063 0.044 
Alhambra Unified 0.057 0.010 
Santa Monica-Malibu Unified 0.055 0.018 
Beardsley Elementary 0.054 0.050 
McSwain Union Elementary 0.052 0.036 
Pleasanton Unified 0.050 0.064 
Arcadia Unified 0.049 0.096 
Albany City Unified 0.048 0.028 
Culver City Unified 0.047 0.087 
Salida Union Elementary 0.045 0.041 
Elk Grove Unified 0.045 0.107 
Alum Rock Union Elementary 0.043 0.210 
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District Name 


Hispanic Student Residual White Student Residual 


Lompoc Unified 0.043 0.081 
San Diego Unified 0.043 0.142 
Moorpark Unified 0.043 0.078 
Wilsona Elementary 0.041 0.182 
San Ysidro Elementary 0.041 0.186 
Grossmont Union High 0.040 0.012 
Upland Unified 0.039 0.031 
Palmdale Elementary 0.038 0.017 
Redding Elementary 0.036 0.059 
Empire Union Elementary 0.036 0.025 
La Mesa-Spring Valley 0.035 0.120 
Golden Valley Unified 0.034 0.061 
Roseville City Elementary 0.032 0.068 
Rescue Union Elementary 0.030 0.021 
Fullerton Elementary 0.028 0.030 
Goleta Union Elementary 0.027 0.130 
Temecula Valley Unified 0.027 0.037 
Pomona Unified 0.025 0.074 
Oceanside Unified 0.024 0.100 
Placerville Union Elementary 0.023 0.035 
Bakersfield City 0.022 0.081 
Palm Springs Unified 0.019 0.009 
Orcutt Union Elementary 0.013 0.051 
Hacienda La Puente Unified 0.013 0.054 
Strathmore Union Elementary 0.013 0.007 
Milpitas Unified 0.009 0.184 
Norwalk-La Mirada Unified 0.009 0.025 
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District Name Hispanic Student Residual White Student Residual 


Robla Elementary 


Torrance Unified 


Victor Elementary 


Sweetwater Union High 


Ventura Unified 


Livingston Union 


Fontana Unified 


Taft City 


Data sources: California Department of Education. (n.d.). California Assessment of Student Performance and Progress 
(CAASPP) results. https://caaspp.cde.ca.gov (accessed 01/05/18); National Center for Education Statistics. (n.d.). Education 
demographic and geographic estimates. https://nces.ed.gov/programs/edge (accessed 01/05/18). 
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